Papers with Linguistics & Cognitive NLP
Using and comparing Rhetorical Structure Theory parsers with rst-workbench (2021.eacl-demos)
Copied to clipboard
| Challenge: | Rhetorical Structure Theory (RST) parsers are usually only trained on English data . |
| Approach: | rst-workbench is a web-based tool that lets users install and use RST parsers. |
| Outcome: | rst-workbench is a web-based tool that lets users run multiple RST parsers simultaneously. |
Item Response Theory for Natural Language Processing (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial introduces the wider NLP community to Item Response Theory (IRT) existing software for fitting IRT models is limited by human-data sized constraints. |
| Approach: | They will introduce IRT and the mathematical foundations which make IRT models. |
| Outcome: | This tutorial aims to introduce the wider NLP community to Item Response Theory and show its benefits for a number of NLP tasks. |
On the Relationship between Zipf’s Law of Abbreviation and Interfering Noise in Emergent Languages (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have shown that emergent languages do not obey ZLA when neural agents play a signaling game. |
| Approach: | They propose to add an explicit penalty on word lengths to a signaling game to simulate a ZLA-like tendency when interfering noises are added to the agents' environment. |
| Outcome: | The proposed model shows that the noise on a speaker is one of the factors for ZLA, while noise on the listener and a channel is not. |
An Item Response Theory Framework for Persuasion (2022.findings-naacl)
Copied to clipboard
| Challenge: | Several studies have considered the audience as a whole or by building separate models for different types of audiences. |
| Approach: | They propose a framework that can represent individual audience members in one model across a diverse set of persuasion tasks. |
| Outcome: | The proposed model performs well on three datasets including a novel dataset in the area of political advocacy. |
Modelling Narrative Elements in a Short Story: A Study on Annotation Schemes and Guidelines (2020.lrec-1)
Copied to clipboard
| Challenge: | Text-processing algorithms that annotate main components of a story are in great need of corpora and well-agreed annotation schemes. |
| Approach: | They propose a model that generalizes a narrative structure in the form of world building elements (characters, time and space) and text worlds themselves and switches between them. |
| Outcome: | The proposed model can be used for annotating narratives in corpora of literary texts, criminal evidence, teaching materials, quests, etc. |
Orthographic Codes and the Neighborhood Effect: Lessons from Information Theory (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has shown that orthography is a core issue in contemporary word reading models. |
| Approach: | They consider the orthographic neighborhood effect as an important control variable in psycholinguistic studies of word reading. |
| Outcome: | The results show that weighting features improves the orthographic neighborhood effect significantly across languages. |
Narrative Theory for Computational Narrative Understanding (2021.emnlp-main)
Copied to clipboard
| Challenge: | a growing body of theoretical work on narrative has been focused on the field of natural language processing . this position paper aims to provide a unifying framework for the computational study of narrative . |
| Approach: | They propose to introduce dominant theoretical frameworks to the NLP community and situate current research within distinct narratological traditions. |
| Outcome: | The proposed framework would allow for new empirical questions and applications in the field of natural language processing. |
Sensala: a Dynamic Semantics System for Natural Language Processing (C18-2)
Copied to clipboard
| Challenge: | SENSALA is an open source logic-based system for formal semantics of natural language. |
| Approach: | They propose to use a lambda calculus with exception handling to handle a wide range of linguistic phenomena, including donkey anaphora, verb phrase anaphorisms, propositional anaphony, presuppositions and implicatures. |
| Outcome: | The framework is based on a lambda calculus with exception handling and uses contexts, continuations, events and dependent types to handle a wide range of linguistic phenomena. |
From Cognitive to Computational Modeling: Text-based Risky Decision-Making Guided by Fuzzy Trace Theory (2022.findings-naacl)
Copied to clipboard
| Challenge: | Fuzzy trace theory explains human risky decision-making by incorporating gists, i.e. fuzzy representations of information which capture only its quintessential meaning. |
| Approach: | They propose a computational framework which combines the effects of the underlying semantics and sentiments on text-based decision-making. |
| Outcome: | The proposed framework can be optimised to predict risky decision-making in groups and individuals. |
Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research (2026.tacl-1)
Copied to clipboard
| Challenge: | Analogical reasoning is an essential aspect of human cognition, says aaron eliotta . eelisa e. sabet: some have argued that analogy is central to the human cognitive experience . |
| Approach: | They summarize key theories about the processes underlying analogical reasoning from the cognitive science literature and relate it to current research in natural language processing. |
| Outcome: | The proposed approaches are relevant for several major challenges in natural language processing, not directly related to analogy solving. |
Looking for Structure in Lexical and Acoustic-Prosodic Entrainment Behaviors (N18-2)
Copied to clipboard
| Challenge: | Entrainment is the tendency of human interlocutors to adapt their behavior to each other to become more similar. |
| Approach: | They propose to use acoustic-prosodic and lexical entrainment to measure speakers' overall entraining behaviors and search for an underlying structure. |
| Outcome: | The proposed method does not find any significant correlations, clusters, or principal components in various entrainment measures. |
Generative Personality Simulation via Theory-Informed Structured Interview (2026.eacl-long)
Copied to clipboard
Pengda Wang, Huiqi Zou, Han Jiang, Hanjie Chen, Tianjun Sun, Xiaoyuan Yi, Ziang Xiao, Frederick L. Oswald
| Challenge: | Personality structured interviews are often lacking in advancing social science research. |
| Approach: | They propose a method to incorporate psychological insights into LLM simulations . they use a measure theory grounded evaluation procedure to evaluate reliability and validity . |
| Outcome: | The proposed method improves human-like heterogeneity in LLM-simulated personality data and predicts personality-related behavioral outcomes. |
Tracing Semantic Variation in Slang (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to slang semantic variation do not account for the semantic variation of sling among different groups of users. |
| Approach: | They propose to use slang semantic variation models to trace the regional identity of a new emerging sling sense given its historical meanings. |
| Outcome: | The proposed models can predict regional identity of emerging slang word meanings from historical sling dictionary entries. |
Modelling Language Acquisition through Syntactico-Semantic Pattern Finding (2023.findings-eacl)
Copied to clipboard
| Challenge: | Usage-based theories of language acquisition have documented the processes by which children acquire language through communicative interaction. |
| Approach: | They propose a method for learning grammars based on similarities and differences in linguistic observations alone. |
| Outcome: | The proposed method is able to learn compositional lexical and item-based constructions of variable extent and degree of abstraction, along with a network of emergent syntactic categories. |
Revisiting Weak-to-Strong Generalization in Theory and Practice: Reverse KL vs. Forward KL (2025.findings-acl)
Copied to clipboard
| Challenge: | Weak-to-strong generalization is a promising approach to guide stronger systems, but its effectiveness is constrained by the inherent imperfections of weak model supervision. |
| Approach: | They propose a theoretically grounded approach that replaces forward KL divergence with reverse KL, which prioritizes high-confidence predictions. |
| Outcome: | The proposed approach replaces forward KL divergence with reverse KL, reducing the influence of unreliable weak supervision. |
No Questions are Stupid, but some are Poorly Posed: Understanding Poorly-Posed Information-Seeking Questions (2025.acl-long)
Copied to clipboard
| Challenge: | When a question is poorly posed, answerers struggle to converge on dominant interpretations, while models attempt comprehensive coverage by addressing many interpretations simultaneously. |
| Approach: | They propose a computational framework to study poorly-posedness of questions by generating spaces of potential interpretations and computing distributions based on interpretations chosen by answerers in the Reddit question thread. |
| Outcome: | The proposed framework analyzes poorly-posed questions using a set of interpretations chosen by human answerers and large language models. |
Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decoding (2026.eacl-long)
Copied to clipboard
| Challenge: | Standard autoregressive decoding in large language models is short-sighted, often failing to find globally optimal reasoning paths due to token-by-token generation process. |
| Approach: | They propose a principled framework that reformulates LLM decoding as a problem of identifying an optimal stochastic process. |
| Outcome: | The proposed framework surpasses state-of-the-art methods in accuracy while significantly improving computational efficiency. |
A Notion of Complexity for Theory of Mind via Discrete World Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) can be used to assess the capabilities of Large Language Models (LLMs) in complex scenarios where social reasoning is required. |
| Approach: | They propose a framework inspired by cognitive load theory to measure the complexity of ToM tasks by a prompting technique that augments the information available to a model with a description of how the environment changes with the agents’ interactions. |
| Outcome: | The proposed framework assesses the complexity of five widely adopted ToM benchmarks and shows that it performs better than other frameworks. |
Modeling the Severity of Complaints in Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Complaining is a speech act used by humans to communicate a negative mismatch between reality and expectations . recent work on modeling complaints in natural language processing (NLP) has focused on distinguishing complaints from non-complaints in social media. |
| Approach: | They propose to classify complaints into various severity levels based on the face-threat that the complainer is willing to undertake and their purpose. |
| Outcome: | The proposed model achieves 55.7 macro F1 on binary complaint classification and 88.2 macro F1. |
ALYMPICS: LLM Agents Meet Game Theory (2025.coling-main)
Copied to clipboard
| Challenge: | Alympics provides a framework for simulating human-like strategic interactions with Large Language Model (LLM) agents. |
| Approach: | They propose a framework utilizing Large Language Models (LLM) agents for empirical game theory research. |
| Outcome: | The proposed framework can be used to study human-like strategic interactions with large language model (LLM) agents in a game on the multi-round auction of scarce survival resources. |
Incorporating Centering Theory into Neural Coreference Resolution (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent years, transformer-based coreference resolution systems have achieved remarkable improvements on the CoNLL dataset. |
| Approach: | They propose to incorporate centering transitions derived from centering theory into a neural coreference model by using a graph. |
| Outcome: | The proposed model improves on pronoun resolution in long documents, formal well-structured text, and clusters with scattered mentions. |
AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)
Copied to clipboard
Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei
| Challenge: | Large language models are increasingly employed to empower autonomous agents to simulate human behavior. |
| Approach: | They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts. |
| Outcome: | The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning. |
Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind (2023.findings-acl)
Copied to clipboard
| Challenge: | Adaptation is a process in human communication by which a speaker tunes its language to that of a listener to achieve communicative success. |
| Approach: | They propose a visual-based referential game between a knowledgeable speaker and a listener with limited visual and linguistic experience to model this adaptation mechanism. |
| Outcome: | The proposed model improves on plug-and-play approaches to controlled language generation without finetuning the speaker’s underlying language model. |
Modeling Human-Like Cognition for Stance Detection: Integrating Intuitive Judgment and Analytical Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have revolutionized stance detection, enabling complex reasoning strategies such as chain-of-thought prompting. |
| Approach: | They propose Cognitive-Driven Stance Detection (CDSD) that integrates fast intuitive judgment and analytical reasoning enhanced by three key modules: attention-based cognitive alignment to compare system focus, uncertainty-aware belief update using Bayesian inference, and self-doubt-triggered counterfactual reasoning for re-evaluation under low consistency or high uncertainty. |
| Outcome: | The proposed method outperforms state-of-the-art methods on SEM16, P-Stance, and VAST. |
Characterizing Idioms: Conventionality and Contingency (2022.acl-long)
Copied to clipboard
| Challenge: | idioms have non-canonical meanings, but non-conventional meanings are contingent on other words . a recent study shows that idiomatic expressions are not homogeneous among idiomas . |
| Approach: | They propose to use a contingency relationship between words in an idiom and non-canonical meanings of words in the idiome. |
| Outcome: | a new study shows that idioms fall at the expected intersection of the two dimensions, but that the dimensions themselves are not correlated. |
One LLM Does Not Simulate All Students: Ability-Aware Student Simulation via Cognitive Diagnosis Guided LLM Assignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods rely on a single high-capacity LLM to represent an entire population of diverse learners. |
| Approach: | They propose an ability-aware student simulation framework that matches students with appropriate LLM backbones through cognitive alignment. |
| Outcome: | The proposed framework significantly reduces simulation bias and outperforms single-model baselines across the entire proficiency spectrum. |
Enhancing the AI2 Diagrams Dataset Using Rhetorical Structure Theory (L18-1)
Copied to clipboard
| Challenge: | Existing annotation schemas for diagrams are based on Rhetorical Structure Theory (RST) paper documents proposed schema, reports on inter-annotator agreement for this task, and discusses use of AI2D-RST for research on multimodality and artificial intelligence. |
| Approach: | They propose to replace the annotation of semantic relations between diagram elements by building on Rhetorical Structure Theory (RST) the paper documents the proposed annotation schema, describes challenges in applying RST to diagrams, and reports on inter-annotator agreement for this task. |
| Outcome: | The proposed schema is based on Rhetorical Structure Theory, which has been used to describe the multimodal structure of diagrams and documents. |
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that transformers can generate both implications of a theory and the natural language proofs that support them. |
| Approach: | They propose a generative model that generates both implications of a theory and natural language proofs that support them. |
| Outcome: | The proposed model generates both implications of a theory and the natural language proofs that support them. |
The Shift from Logic to Dialectic in Argumentation Theory: Implications for Computational Argument Quality Assessment (2025.coling-main)
Copied to clipboard
| Challenge: | In the field of computational argument quality assessment, logic and dialectic are essential dimensions used to measure the quality of argumentative texts. |
| Approach: | They propose to separate logic and dialectic as quality dimensions in computational argument quality assessment . they propose to use dialectical considerations to improve the quality of argumentative texts . |
| Outcome: | The proposed method can benefit argument theory and argument analysis by separating the two quality dimensions. |
A Review of Incorporating Psychological Theories in LLMs (2026.eacl-long)
Copied to clipboard
Zizhou Liu, Ziwei Gong, Lin Ai, Zheng Hui, Run Chen, Colin Wayne Leach, Michelle R. Greene, Julia Hirschberg
| Challenge: | a holistic review systematically integrating psychology across the LLM lifecycle remains missing. |
| Approach: | They examine how psychological theories can inform stages of LLM development . they highlight current trends and gaps in how psychological theory is applied . |
| Outcome: | The authors highlight current trends and gaps in how psychological theories are applied . they argue that psychological insights have shaped pivotal NLP breakthroughs . |
Comparative Probing of Lexical Semantics Theories for Cognitive Plausibility and Technological Usefulness (2020.coling-main)
Copied to clipboard
| Challenge: | lexical semantics theories advocate that the meaning of words is represented as an inference graph, a feature mapping or a cooccurrence vector. |
| Approach: | They propose to probe different lexical semantics theories for their levels of cognitive plausibility and technological usefulness. |
| Outcome: | The proposed approach is superior to the other approaches, the authors argue . they show that the feature-based approach is more useful and more plausible than the other ones . |
End-to-End Sequential Metaphor Identification Inspired by Linguistic Theories (P19-1)
Copied to clipboard
| Challenge: | Existing sequence tagging models do not explicitly exploit linguistic theories of metaphor identification. |
| Approach: | They propose to exploit linguistic theories of metaphor identification in deep neural networks to improve model performance. |
| Outcome: | The proposed models achieve state-of-the-art in end-to-end metaphor identification on three datasets. |
Navigating the Infinite Dynamic Web Space: Effective In-Context Exploration via Cognitive Multi-Agent Collaboration (2026.eacl-long)
Copied to clipboard
Guozhao Mo, Yanjiang Liu, Yafei Shi, Jiawei Chen, Yang Li, Yaojie Lu, Hongyu Lin, Ben He, Le Sun, Bo Zheng, Xianpei Han
| Challenge: | Existing methods for dynamic web navigation rely on greedy strategies or value estimation, struggle to achieve effective backtracking and are heavily dependent on proprietary models. |
| Approach: | They propose a cognitive multi-agent collaboration framework that enhances cyberspace exploration capability through In-Context Exploration. |
| Outcome: | The proposed framework surpasses the proprietary model Claude-3.5 Sonnet on the WebArena benchmark. |
Detection and Positive Reconstruction of Cognitive Distortion Sentences: Mandarin Dataset and Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have investigated the application of NLP models in English for each stage of this process. |
| Approach: | They propose a Positive Reconstruction Framework based on broaden-and-build theory to address and reframe negative thoughts through a positive reinterpretation. |
| Outcome: | The proposed framework is based on broaden-and-build theory and can detect cognitive distortions and suggest a positive reframe in Mandarin. |
A Theory Guided Scaffolding Instruction Framework for LLM-Enabled Metaphor Reasoning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for metaphor detection and reasoning struggle to explain the underlying reasoning process behind the metaphorical/literal judgment. |
| Approach: | They propose a Theory guided Scaffolding Instruction framework that instructs an LLM to infer the underlying reasoning process of metaphor detection guided by metaphor theories for the first time. |
| Outcome: | The proposed method significantly outperforms both the LLM-based reasoning methods and the SOTA methods in metaphor detection. |
CDS: Data Synthesis Method Guided by Cognitive Diagnosis Theory (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but the complexity of emerging tasks and higher performance demands highlight the need for continuous improvement. |
| Approach: | They propose a method that refines evaluation results and characterizes model profiles at the knowledge component level. |
| Outcome: | The proposed method improves performance across multiple benchmarks and academic exams. |
Agents generalize to novel levels of abstraction by using adaptive linguistic strategies (2025.findings-acl)
Copied to clipboard
| Challenge: | Abstract: Abstracts are fundamental to building well-generalizing models. |
| Approach: | They propose to use a concept-level reference game to generalize concepts . they find that agents can learn robust concepts based on which they can generalize . |
| Outcome: | The proposed model can generalize from generic to very specific concepts, while reusing many messages from training. |
Is a Peeled Apple Still Red? Evaluating LLMs’ Ability for Conceptual Combination with Property Type (2025.naacl-long)
Copied to clipboard
| Challenge: | Conceptual combination is a cognitive process that merges basic concepts, enabling the creation of complex expressions. |
| Approach: | They propose to use a Conceptual Combination with Property Type dataset to evaluate LLMs for conceptual combination thoroughly. |
| Outcome: | The proposed method improves performance in all generative tasks. |
(Re)construing Meaning in NLP (2020.acl-main)
Copied to clipboard
| Challenge: | a new paper explores the role of linguistic choices in interpreting information in natural language . linguistic choice is a way of expressing information, but it is not the meaning of an utterance, authors argue . |
| Approach: | They propose to define construal as a way of conceptualizing or construing information . they propose to use this concept to develop theoretical and practical work in NLP . |
| Outcome: | The proposed study explores how construal can inform theoretical and practical work in NLP. |
Modeling Conceptual Attribute Likeness and Domain Inconsistency for Metaphor Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Metaphor detection aims to distinguish between metaphorical and literal expressions in text. |
| Approach: | They propose an attribute likeness and domain inconsistency learning framework for wordpair metaphor detection based on conceptual metaphor theory . they model attribute likeity with an attribute siamese network and devise a domain contrastive learning strategy to learn semantic inconsistentness of concepts in source and target domains . |
| Outcome: | The proposed framework outperforms existing word-pair and token-level methods on four datasets. |
Neural reality of argument structure constructions (2022.acl-long)
Copied to clipboard
| Challenge: | lexicalist linguistic theories assume argument structure is predictable from meaning of verbs . construction grammarians propose argument structure constructions distinct from verbs. |
| Approach: | They adapt psycholinguistic studies to probe for the existence of argument structure constructions in Transformer-based language models. |
| Outcome: | The proposed method could be used to probe argument structure constructions in LMs . the study shows that LM learners prefer grouping by construction over verb grouping . |
Annotating High-Level Structures of Short Stories and Personal Anecdotes (L18-1)
Copied to clipboard
| Challenge: | Existing theories for narrative structures have been challenging to operationalize . authors present an annotation scheme to help computer systems understand stories better . |
| Approach: | They propose to consolidate and extend existing narratological theories and an annotation scheme . they will support an approach that enables systems to intelligently sustain complex communications with humans . |
| Outcome: | The proposed method consolidates and extends existing narratological theories . it will support an approach that enables systems to intelligently sustain complex communications with humans . |
Centering-based Neural Coherence Modeling with Hierarchical Discourse Segments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior studies of coherence focused on identifying semantic relations between adjacent sentences. |
| Approach: | They propose a coherence model which takes discourse structural information into account without relying on human annotations. |
| Outcome: | The proposed model performs state-of-the-art on automated essay scoring and assessing writing quality tasks. |
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons (2023.acl-long)
Copied to clipboard
Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi, Prithviraj Ammanabrolu
| Challenge: | Existing dialogue agents, while able to produce human-like responses, often do not model goal-driven and grounded language interactions. |
| Approach: | They propose to decompose and model teacher-student natural language interactions into (1) the DM’s intent to guide players toward a given goal; (2) the dm’s guidance utterance to the players expressing this intent; (3) a theory-of-mind model that anticipates the players’ reaction to the guidance one turn into the future. |
| Outcome: | The proposed task is based on a goal-driven and grounded environment with a teacher-student interaction model and theory-of-mind model. |
Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work assumes that events are sequentially arranged in a script, while this assumption leads to linear generation that is far from sufficient for comprehensively acquiring the representation about how events are organized towards a task goal. |
| Approach: | They propose to extend goal-oriented Script Generation task from the perspective of cognitive theory by incorporating subgoals into hierarchical script generation. |
| Outcome: | The proposed task is based on a new dataset and human evaluation metrics. |
Cooperative or Competitive? Understanding the Interaction between Attention Heads From A Game Theory Perspective (2025.acl-long)
Copied to clipboard
| Challenge: | a number of attention-based large language models (LLMs) focus on individual head contributions, but the precise interaction mechanisms between attention heads remain poorly understood. |
| Approach: | They propose a game-theoretic attention calibration method that uses the Harsanyi dividend . they selectively retain heads demonstrating significant cooperative gains and apply fine-grained adjustments to remaining heads . |
| Outcome: | The proposed framework is based on the Harsanyi dividend, a concept from cooperative game theory. |
Article and Comment Frames Shape the Quality of Online Comments (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on predicting comment toxicity or quality, but it ignores audience reactions. |
| Approach: | They propose a frame-aware system to mitigate unhealthy discourse . they analysed 1M comments across 2.7K news articles . |
| Outcome: | The proposed system can mitigate unhealthy discourses by analyzing 1M comments across 2.7K news articles. |
TuringQ: Benchmarking AI Comprehension in Theory of Computation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | TuringQ is the first benchmark designed to evaluate the reasoning capabilities of large language models (LLMs) in the theory of computation. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in the theory of computation. |
| Outcome: | The proposed system shows competitive accuracy when compared to human evaluation. |
Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory (2024.findings-emnlp)
Copied to clipboard
Suyeon Lee, Sunghwan Kim, Minju Kim, Dongjin Kang, Dongil Yang, Harim Kim, Minseok Kang, Dayi Jung, Min Kim, Seungbeen Lee, Kyong-Mee Chung, Youngjae Yu, Dongha Lee, Jinyoung Yeo
| Challenge: | Existing models that use large language models are not available due to ethical concerns, and data privacy concerns are a concern. |
| Approach: | They propose a multi-turn dialogue dataset that emulates real-life counseling interactions using the goal-oriented approach of Cognitive Behavioral Therapy (CBT). |
| Outcome: | The proposed model outperforms other models in counseling skills, highlighting its effectiveness and potential as a counseling agent. |
Linguistic Alignment Predicts Learning in Small Group Tutoring Sessions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cognitive science offers rich theories of learning and communication, yet these are often difficult to operationalize at scale. |
| Approach: | They investigate linguistic alignment in a longitudinal dataset of real-world tutoring interactions and associated student test scores. |
| Outcome: | The proposed method can be applied to real-world tutoring interactions and student test scores. |
CPT-Agent: A Cognitive Process Theory-driven Framework for Student Simulation in Writing Development (2026.acl-long)
Copied to clipboard
| Challenge: | Existing LLMs model overly capable learners who over-apply feedback, resulting in pedagogically implausible behavior. |
| Approach: | They propose a framework that decouples cognitive ability from writing proficiency and models their interaction during writing and revision. |
| Outcome: | The proposed model produces distinguishable proficiency levels and is consistent with instructional theories. |
Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground (2024.findings-acl)
Copied to clipboard
Adil Soubki, John Murzaku, Arash Yousefi Jordehi, Peter Zeng, Magdalena Markowska, Seyed Abolghasem Mirroshandel, Owen Rambow
| Challenge: | Existing benchmarks for theory of mind (ToM) use synthetic data, which can misalign with human behavior. |
| Approach: | They propose a question-answer benchmark based on naturally occurring spoken dialogs to evaluate theory of mind capabilities of language models. |
| Outcome: | The proposed dataset shows that LMs struggle to demonstrate theory of mind (ToM) . |
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations for theory of mind (ToM) use passive narratives that lack interactivity. |
| Approach: | They propose a benchmark to stress-test ToM within information-asymmetric conversational contexts via question answering. |
| Outcome: | The proposed benchmark is challenging for state-of-the-art language models, which perform significantly worse than humans even with chain-of thought reasoning or fine-tuning. |
LeadEmpathy: An Expert Annotated German Dataset of Empathy in Written Leadership Communication (2024.lrec-main)
Copied to clipboard
| Challenge: | Empathetic leadership communication is associated with a wide range of positive individual and organizational outcomes. |
| Approach: | They propose an expert-annotated german dataset for modeling empathy in written leadership communication that uses a theory-based coding scheme to model cognitive and affective empathy in asynchronous communication. |
| Outcome: | The proposed model can be applied to produce high-quality, multidimensional empathy ratings in current and future applications. |
LexiVault: A Repository for Psycholinguistic Lexicons of Lesser-studied Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | LexiVault is an open-source web tool with annotated lexicons and rich retrieval capabilities primarily developed for, but not restricted to, the support of psycholinguistic research . |
| Approach: | They propose to use LexiVault to design stimuli for low-resource languages with annotated lexicons and rich retrieval capabilities. |
| Outcome: | The LexiVault tool is designed to be user friendly and accommodate incremental growth of new and existing low-resource language lexicons in the system while abstracting programming complexity to foster more interest from the psycholinguistics community in exploring low-rsource languages. |
Towards a Similarity-adjusted Surprisal Theory (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that surprisal theory ignores the possibility of similarity between words and treats them as distinct entities. |
| Approach: | They propose a new measure of comprehension effort called information value that accounts for communicative equivalences between possible continuations. |
| Outcome: | The proposed measure of comprehension effort is based on the diversity index of the diversity of communicative units. |
Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld’s Episode Theory (2025.emnlp-main)
Copied to clipboard
Ming Li, Nan Zhang, Chenrui Fan, Hong Jiao, Yanbin Fu, Sydney Peters, Qingshu Xu, Robert Lissitz, Tianyi Zhou
| Challenge: | Large Reasoning Models (LRMs) generate extensive chain-of-thought reasoning, but we lack a principled framework for understanding how these thoughts are structured. |
| Approach: | They propose a method to analyze the reasoning traces of Large Reasoning Models using Schoenfeld’s Episode Theory. |
| Outcome: | The proposed framework provides a theoretically grounded methodology for interpreting LRM cognition and enables future work on more controllable and transparent reasoning systems. |
Machine Theory of Mind Needs Machine Validation (2025.findings-acl)
Copied to clipboard
| Challenge: | In recent years there has been an explosion of interest in studying the extent to which language models (LMs) display a theory of mind (ToM) despite the growth of evaluation tools, the extent of evidence for ToM remains unclear. |
| Approach: | They conduct a survey of 16 recent studies aimed at measuring ToM in language models and found that only half do so for patterns only a machine might exploit. |
| Outcome: | The results show that the datasets that show high LM performance on ToM tasks are easier than their peers, likely due to the presence of spurious patterns in the data. |
Déjà Vu? Decoding Repeated Reading from Eye Movements (2025.acl-long)
Copied to clipboard
| Challenge: | In many daily situations we read the same text more than once. |
| Approach: | They propose a strategy for enhancing feature-based and neural models by generating machine generated eye movements from a cognitive model. |
| Outcome: | The proposed model improves on the previous model and enables better characterization of the role of memory in repeated reading. |
Is a cute puyfred cute? Context-dependent form-meaning systematicity in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | valence is encoded in meaningful ways in large language models and in some LLMs, pseudowords affect the representation of whole sentences similarly to words. |
| Approach: | They investigate how LLMs represent valence, a key semantic attribute, and how they deal with contextualisation of pseudowords in sentences. |
| Outcome: | The results show that the models represent valence, a key semantic attribute, in sentences and in context, and that they handle the contextualisation of pseudowords differently. |
The Linearity of the Effect of Surprisal on Reading Times across Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a large amount of insight into human language processing can be gleaned by studying word-by-word processing difficulty. |
| Approach: | They extend the study by examining eyetracking corpora of seven languages . they find evidence for superlinearity in some languages, but highly sensitive to language models . |
| Outcome: | The study extends existing studies on english to Danish, Dutch, English, German, Japanese, Mandarin, and Russian. |
ToMELP: A Theory-of-Mind Benchmark for Route-Controlled Persuasion under the Elaboration Likelihood Model (2026.findings-acl)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is widely regarded as central to effective persuasion, yet existing evaluations fail to capture the infer–apply loop that arises in real-world dialogue. |
| Approach: | They propose a benchmark that conditions on the audience persona p and the Elaboration Likelihood Model (ELM) route r within persuasive conversations. |
| Outcome: | The proposed model can model the interlocutor's mental states over multiple turns and adapt strategy and tone accordingly. |
Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human States (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks assess basic Theory of Mind abilities but neglect temporal evolution of mental states in real-world social contexts. |
| Approach: | They propose a benchmark specifically designed to evaluate Large Language Models' ability to understand and track the temporal progression of mental states across interconnected scenarios. |
| Outcome: | The proposed benchmarks underperform humans by 44.7% and show that they can model the dynamic nature of human mental states better than existing models. |
M2PA: A Multi-Memory Planning Agent for Open Worlds Inspired by Cognitive Theory (2025.findings-acl)
Copied to clipboard
YanfangZhou YanfangZhou, Xiaodong Li, Yuntao Liu, Yongqiang Zhao, Xintong Wang, Zhenyu Li, Jinlong Tian, Xinhai Xu
| Challenge: | Open-world planning poses a challenge due to complex environments and task diversity . recent work shows that large language models (LLMs) lack the ability to connect to agents' experiences . |
| Approach: | They propose an open-world multi-memory planning agent that combines large language models with human-like multi-mesh systems to leverage their strengths. |
| Outcome: | The proposed agent outperforms state-of-the-art agents on 50 Minecraft tasks in zero-shot learning. |
TactfulToM: Do LLMs have the Theory of Mind ability to understand White Lies? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies explore Large Language Models’ (LLMs) performance on Theory of Mind (ToM) reasoning tasks, but research on ToM abilities that require more nuanced social context is limited, such as white lies. |
| Approach: | They propose a novel English benchmark to evaluate Large Language Models’ ability to understand white lies within real-life conversations and reason about prosocial motivations behind them. |
| Outcome: | The proposed model outperforms state-of-the-art models on ToM reasoning tasks and reveals significant gaps between humans and LLMs. |
SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, estimating item difficulties requires real students to respond to items . a cold-start approach cannot be applied to previously unseen items either . |
| Approach: | They propose a method for aligning simulated students with instructed ability to predict difficulty of open-ended items. |
| Outcome: | The proposed method outperforms existing methods on two real-world student responses. |
Specifying Genericity through Inclusiveness and Abstractness Continuous Scales (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a pilot study, we created a small but crucial annotated dataset of 324 sentences, demonstrating the framework’s effectiveness in capturing nuanced aspects of genericity. |
| Approach: | They propose a framework for fine-grained modeling of noun phrases' genericity in natural language using a small but crucial annotated dataset of 324 sentences. |
| Outcome: | The proposed framework can be used to model genericity of noun phrases in natural language and can be easily compared with existing binary annotations. |
Under the Shadow of Babel: How Language Shapes Reasoning in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | linguistic relativity suggests that the structure of language shapes cognitive patterns . large language models internalize the habitual logical structures embedded in different languages, authors say . |
| Approach: | a study introduces a bilingual dataset for causal reasoning in Chinese and English. |
| Outcome: | a new study shows that large language models internalize reasoning biases shaped by language . the model internalizes language-specific preferences and rigidly applies them to atypical inputs, the study shows . |
ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback (2026.findings-acl)
Copied to clipboard
| Challenge: | ProToM provides targeted, context-sensitive feedback to individual agents, achieving a higher success rate, shorter task completion times, and is consistently preferred by human users. |
| Approach: | They propose a Theory of Mind-informed facilitator that provides targeted, context-sensitive feedback to individual agents. |
| Outcome: | The proposed system provides targeted, context-sensitive feedback to promote prosocial behaviour, even when not directly aligned with one’s own goals. |
Why Are We Moral? An LLM-based Agent Simulation Approach to the Study of Moral Evolution (2026.acl-long)
Copied to clipboard
Zhou Ziheng, Huacong Tang, Mingjie Bi, Wanying He, Fang Sun, Yizhou Sun, Ying Nian Wu, Demetri Terzopoulos, Yipeng Kang, Fangwei Zhong
| Challenge: | Existing models of moral evolution must abstract away cognitive processes . et al. (2017): evolution of morality presents a puzzle: natural selection favors selfish . |
| Approach: | They propose an LLM-based agent simulation framework that manipulates cognitive factors to understand moral evolution. |
| Outcome: | The proposed model exploits cognitive realism to explore moral evolution in a hunter-gatherer society. |
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for Theory of Mind (ToM) focus on whether agents have correct beliefs about others. |
| Approach: | They propose to evaluate Theory of Mind (ToM) capabilities in Large Language Models (LLMs) they propose to use the theory of mind to determine whether and how to invoke ToM . |
| Outcome: | The proposed frameworks can be used to evaluate the performance of large language models (LLMs) in biological agents. |
Agentic-ToM: Cognition-Inspired Agentic Processing For Enhancing Theory of Mind Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current models struggle with reasoning about others’ perspectives, limiting their ability to attribute mental states to oneself and others. |
| Approach: | They propose to embed psychologically-grounded functions into LLMs to enable them to attribute mental states to oneself and others, known as Theory of Mind. |
| Outcome: | The proposed approach outperforms baselines on three ToM datasets without task-specific modifications. |
From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing statistical methods for evacuation decision prediction fail to capture complex and diverse behavioral logic of different individuals. |
| Approach: | They propose a Large Language Model (LLM)-based framework that integrates behavioral theories and models to streamline the Chain-of-Thought reasoning and integrates with memory-based Reinforcement Learning module to provide accurate evacuation decision prediction and understanding. |
| Outcome: | The proposed framework improves on three post-wildfire survey datasets with strong cross-event generalizability over existing models. |
CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models (2026.acl-long)
Copied to clipboard
Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, Qian Zhang, Yi Zeng
| Challenge: | Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions. |
| Approach: | They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators . |
| Outcome: | The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models. |
A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly utilized in autonomous decision-making, where they sample options from vast action spaces. |
| Approach: | They propose to use heuristics to sample LLMs to identify a prescriptive component and a descriptive component to represent a concept. |
| Outcome: | The proposed model is compared with human decision-making models in public health, economic trends and other real-world domains to show that it is biased. |
ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integrated into agentic frameworks to assist individual users in completing diverse tasks. |
| Approach: | They propose a simulation environment with a plug-and-play proactive AI mediator . they use a socio-cognitive evaluation framework to measure consensus changes, intervention latency, mediator effectiveness and intelligence. |
| Outcome: | The proposed model outperforms a generic baseline in multi-party negotiation scenarios while being 77% faster in response. |
On the Proper Treatment of Units in Surprisal Theory (2026.acl-long)
Copied to clipboard
| Challenge: | empirical work often leaves the notion of a unit underspecified . empirical work has sought to characterize the processing difficulty comprehenders experience . |
| Approach: | They propose a framework for reasoning about surprisal over arbitrary unit inventories . they argue that surprises should be explicit and treat tokenization as implementation detail . |
| Outcome: | The proposed framework disentangles the models' definitions and the regions of interest and treats tokenization as an implementation detail rather than a scientific primitive. |
ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Theory of Mind (ToM) benchmarks focus on text-only or dyadic interactions, but to address this gap, we propose ToM-SSI: a new benchmark specifically designed to test ToM capabilities in environments rich with social interactions and spatial dynamics. |
| Approach: | They propose to use the Sally-Anne test to test ToM capabilities in environments rich in social interactions and spatial dynamics. |
| Outcome: | The proposed model captures a wider range of social cognition than existing models and demonstrates that existing models are still limited in these new tasks. |
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal (2026.acl-long)
Copied to clipboard
| Challenge: | Surprisal theory claims that difficulty of sentences increases linearly with surprise . a neural LM that can explain garden-path effects cannot be built, says a new study . |
| Approach: | They propose to fine-tune neural LMs to better align surprisal-based reading-time estimates with actual reading times. |
| Outcome: | a new study shows that fine-tuned neural LMs do not overfit on held-out items . the results show that they improve predictive power for human reading times . |
ReDepress: A Cognitive Framework for Detecting Depression Relapse from Social Media (2025.emnlp-main)
Copied to clipboard
Aakash Kumar Agarwal, Saprativa Bhattacharjee, Mauli Rastogi, Jemima S. Jacob, Biplab Banerjee, Rashmi Gupta, Pushpak Bhattacharyya
| Challenge: | Almost 50% of depression patients face the risk of going into relapse. |
| Approach: | They propose to validate a social media dataset on depression relapse using cognitive theories of depression. |
| Outcome: | The first clinically validated social media dataset focused on depression relapse comprises 204 Reddit users annotated by mental health professionals. |
HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns (2026.acl-long)
Copied to clipboard
Xintao Wang, Jian Yang, Weiyuan Li, Rui Xie, Jen-tse Huang, Jun Gao, Shuai Huang, Yueping Kang, Yuanli Guo, Hongwei Feng, Yanghua Xiao
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and generation, serving as the foundation for advanced persona simulation and Role-Playing Language Agents (RPLAs). |
| Approach: | They propose a framework that treats psychological patterns as interacting causal forces and synthesizes 113 scenarios where 2-5 patterns reinforce, conflict, or modulate each other. |
| Outcome: | The proposed framework outperforms Qwen3-32B on multi-pattern dynamics despite 4 fewer parameters. |
GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs (2026.acl-long)
Copied to clipboard
Weidong Tang, Jierui Li, Yueling Hou, Zihan Mei, Can Zhang, Xinyan Wan, Zhiyuan Liang, Pengfei Zhou, Yang You, Wangbo Zhao
| Challenge: | Existing models for general intelligence fail to model how mental states interact and crystallize into group-level outcomes. |
| Approach: | They propose a multimodal benchmark for group-level Theory of Mind (ToM) to probe nonlinear collective behavior. |
| Outcome: | The proposed model performs significantly below human levels, exposing blind spots in modeling social structures and nonlinear collective behavior. |
Towards A Scanpath-Conditioned Surprisal Theory: Modeling Reader Information States (2026.acl-long)
Copied to clipboard
| Challenge: | Standard surprisal is computed from the linear text prefix, but human reading is non-linear and memory constrained. |
| Approach: | They propose a formulation of surprisal conditioned on a reader-specific accessible information state given by the scanpath history and memory dynamics rather than by the written prefix alone. |
| Outcome: | The proposed approach improves on eye-tracking measures on the written prefix and on eye movement data on human reading. |
How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that role models influence morality, but they are not uniformly interpreted and appropriated in groups with heterogeneous motivations. |
| Approach: | They build a multi-agent simulation where agents with diverse intrinsic drives interact and adapt through a four-stage cognitive loop. |
| Outcome: | The proposed model can significantly reshape morality of agents with diverse intrinsic drives . the simulations show that identity-driven conformity can substantially reshaped initial dispositions . |
Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation (2026.findings-acl)
Copied to clipboard
Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai
| Challenge: | Recent studies have applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. |
| Approach: | They propose four alignment metrics and use Mann–Whitney U tests to compare agents’ style-switching behavior with financial theory. |
| Outcome: | The proposed model is only partially consistent with financial theory. |
Can Small LLMs Learn a Robust Theory of Mind via RLVR? Investigating Generalization through the False-Belief Task (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated emergent capabilities in complex reasoning, largely spurred by rule-based Reinforcement Learning (RL) techniques applied during post-training. |
| Approach: | They evaluate whether small-scale LLMs can acquire a robust and generalizable Theory of Mind (ToM) capability through RL with verifiable rewards. |
| Outcome: | The proposed model performs well on in-distribution tasks but fails to transfer to unseen ToM tasks with different characteristics. |
Does Theory of Mind Improvement Really Benefit Human-AI Interactions? Empirical Findings from Interactive Evaluations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks measure ToM capability improvement through story-reading, multiple-choice questions from a third-person perspective, while ignoring the first-person, dynamic nature of human-AI interactions. |
| Approach: | They propose a new paradigm of interactive ToM evaluation with both perspective and metric shifts. |
| Outcome: | The proposed approach improves the performance of four representative LLM enhancement techniques using real-world datasets and a user study. |